Input apparatus, input method, and recording medium recording input program

ABSTRACT

An input apparatus includes: an operation position determination device that determines an operation position of a gesture operation by a user; a movement vector calculator that calculates a movement vector at an input position on the basis of a movement amount of the operation position when the operation position moves; an input processor that executes first input processing at the input position at the time when a first gesture operation is detected, and executes second input processing at the input position at the time when a second gesture operation is detected; and a movement vector corrector that corrects the movement vector in the case where a change from the first gesture operation to the second gesture operation is determined.

INCORPORATION BY REFERENCE

This application is based upon and claims the benefit of priority fromthe corresponding Japanese Patent Application No. 2021-100090 filed onJun. 16, 2021, the entire contents of which are incorporated herein byreference.

BACKGROUND

The present disclosure relates to an input apparatus that accepts inputthrough a gesture operation by a user on a display screen, an inputmethod, and a recording medium that records an input program.

Conventionally, an input apparatus that enables input (a screenoperation) through a gesture operation on a display screen of a displaypanel has been known. For example, the following interface device hasbeen known. When a user puts one finger up toward equipment, an icon ofnumber “1” is shown on a display, and display of a TV as a first item ona menu is highlighted. When the user puts two fingers up, the icon ofnumber “2” is shown on the display, and display of a network as a seconditem on the menu is highlighted. When the user keeps the same hand shapefor a certain period of time, the first item or the second item on themenu is selected.

Here, for example, the input apparatus identifies a position on thedisplay screen (an input position) that is instructed through thegesture operation by the user's hand on the basis of a position ofhis/her hand (an operation position). For example, the input apparatusidentifies, as the input position, a position of the user's fingerjoint, a center (center of gravity) position of an entire region wherethe user's hand during the gesture operation is projected, or the like.However, this method causes such a problem that the input position ischanged in a direction unintended by the user in the case where the userchanges his/her operation from an operation to open his/her hand (afirst gesture operation) to an operation to close (clench) his/her hand(a second gesture operation), for example.

SUMMARY

An object of the present disclosure is to provide an input apparatusthat detects a gesture operation by a user, executes input processingwith respect to an input position on a display screen, and can preventthe input position from being changed by a change in the gestureoperation by the user, an input method, and a recording medium thatrecords an input program.

An input apparatus according to one aspect of the present disclosure isan input apparatus that detects a gesture operation by a user andexecutes input processing with respect to an input position on a displayscreen, and includes: a gesture operation detector that detects thegesture operation by the user; an operation position determinationdevice that determines an operation position of the gesture operationdetected by the gesture operation detector; a movement vector calculatorthat calculates a movement vector to move the input position on thebasis of a movement amount of the operation position when the operationposition moves, the movement vector including a movement direction and amovement amount; an input processor that executes first input processingat the input position at the time when the gesture operation detectordetects a first gesture operation by the user, and executes second inputprocessing at the input position at the time when the gesture operationdetector detects a second gesture operation by the user; a gestureoperation change determination device that determines a change from thefirst gesture operation to the second gesture operation; and a movementvector corrector that corrects the movement vector in the case where thegesture operation change determination device determines the change fromthe first gesture operation to the second gesture operation.

An input method according to another aspect of the present disclosure isan input method for detecting a gesture operation by a user andexecuting input processing with respect to an input position on adisplay screen, and causes one or plural processors to execute: gestureoperation detection to detect the gesture operation by the user;operation position determination to determine an operation position ofthe gesture operation, which is detected in the gesture operationdetection; movement vector calculation to calculate a movement vector tomove the input position on the basis of a movement amount of theoperation position when the operation position moves, the movementvector including a movement direction and a movement amount; inputtingto execute first input processing at the input position at the time ofdetecting a first gesture operation by the user in the gesture operationdetection and execute second input processing at the input position atthe time of detecting a second gesture operation by the user in thegesture operation detection; gesture operation change determination todetermine a change from the first gesture operation to the secondgesture operation; and movement vector correction to correct themovement vector when determining the change from the first gestureoperation to the second gesture operation in the gesture operationchange determination.

A recording medium according to yet another aspect of the presentdisclosure is a recording medium that records an input program fordetecting a gesture operation by a user and executing input processingwith respect to an input position on a display screen, and causes one orplural processors to execute: gesture operation detection to detect thegesture operation by the user; operation position determination todetermine an operation position of the gesture operation, which isdetected in the gesture operation detection; movement vector calculationto calculate a movement vector to move the input position on the basisof a movement amount of the operation position when the operationposition moves, the movement vector including a movement direction and amovement amount; inputting to execute first input processing at theinput position at the time of detecting a first gesture operation by theuser in the gesture operation detection and execute second inputprocessing at the input position at the time of detecting a secondgesture operation by the user in the gesture operation detection;gesture operation change determination to determine a change from thefirst gesture operation to the second gesture operation; and movementvector correction to correct the movement vector when determining thechange from the first gesture operation to the second gesture operationin the gesture operation change determination.

According to the present disclosure, it is possible to provide the inputapparatus that detects the gesture operation by the user, executes theinput processing with respect to the input position on the displayscreen, and can prevent the input position from being changed in adirection unintended by the user by the change in the gesture operationby the user, an input method, and a recording medium that records aninput program.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription with reference where appropriate to the accompanyingdrawings. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a displayapparatus according to an embodiment of the present disclosure.

FIG. 2 is a schematic view illustrating an example of a virtualoperation plane in the display apparatus according to the embodiment ofthe present disclosure.

FIG. 3 is a view illustrating an example of an input operation in aconventional display apparatus.

FIG. 4A is a view illustrating an example of an operation position ofthe input operation in the conventional display apparatus.

FIG. 4B is a view illustrating another example of the operation positionof the input operation in the conventional display apparatus.

FIG. 5 is a view illustrating another example of the input operation inthe conventional display apparatus.

FIG. 6 is a view illustrating a circumstance where a gesture operationand a display mode of an input operation icon are changed in the displayapparatus according to the embodiment of the present disclosure.

FIG. 7 is a flowchart for explaining an example of a procedure ofdisplay control processing that is executed in the display apparatusaccording to the embodiment of the present disclosure.

FIG. 8 is a view illustrating an example of a method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 9 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 10 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 11 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 12 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 13 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 14 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 15 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

FIG. 16 is a view illustrating an example of the method for setting thevirtual operation plane in the display apparatus according to theembodiment of the present disclosure.

DETAILED DESCRIPTION

A description will hereinafter be made on an embodiment of the presentdisclosure with reference to the accompanying drawings. The followingembodiment is an example that embodies the present disclosure, and doesnot intend to limit the technical scope of the present disclosure.

As illustrated in FIG. 1 , a display apparatus 1 according to theembodiment of the present disclosure includes a controller 11, a storage12, a display panel 13, an operation acceptor 14, and a motion sensor15. FIG. 2 is a schematic view of the display apparatus 1. The motionsensor 15 is installed on top of the display panel 13 to detect an inputoperation by a user.

The display apparatus 1 accepts the non-contact input operation (forexample, a gesture operation) by the user for a display screen 13A. Forexample, when detecting the input operation by the user on a virtualoperation plane R2, the display apparatus 1 executes input processingthat corresponds to the input operation by the user for the displayscreen 13A. For example, in the case where the user performs anoperation to touch a predetermined position of the virtual operationplane R2, the display apparatus 1 detects a position (an inputposition), which corresponds to the touch position (an operationposition) on the virtual operation plane R2, on the display screen 13Aand accepts touch input. In addition, the display apparatus 1 detectsthe gesture operation by the user and executes the input processing withrespect to the input position on the display screen 13A. The displayapparatus 1 is an example of the input apparatus in the presentdisclosure. A description will hereinafter be made on a specificconfiguration of the display apparatus 1.

For example, the motion sensor 15 includes two cameras and threeinfrared LEDs, and detects the input operation by the user within apredetermined detection range. The motion sensor 15 outputs detectioninformation to the controller 11. The detection information includesposition coordinates (an X-coordinate, a Y-coordinate, and aZ-coordinate) of a detection target (for example, the user's hand, theuser's fingertip, a stylus pen, a pointer, or the like) with the motionsensor 15 being a reference. For example, the motion sensor 15 candetect the back (palm) of the user's hand (right hand RH or left handLH) and joints of the user's fingers as well as skeletons, angles,positions, and the like of the user's hand and fingers. A well-knowntechnique can be applied to the motion sensor 15. The motion sensor 15and the cameras are an example of the imager in the present disclosure.

The display panel 13 is a display that shows an image, and is aliquid-crystal display, for example. The operation acceptor 14 is anoperation device including a mouse and a keyboard, for example.Alternatively, the operation acceptor 14 may constructed of a touchpanel.

The storage 12 is a non-volatile storage such as a hard disk drive (HDD)or a solid state drive (SSD) that stores various types of information.More specifically, the storage 12 stores data such as operation regioninformation D1 and virtual operation plane information D2.

The operation region information D1 is information on an operationregion R1 in the display screen 13A of the display panel 13. Theoperation region R1 is, in the display screen 13A, a region where theuser can perform the input operation via the virtual operation plane R2,that is, a region that can accept the input operation by the user. Theoperation region R1 may be set for an entire region of the displayscreen 13A or may be set for a partial region of the display screen 13A.For example, in the case where the entire region of the display screen13A is set as the operation region R1, the operation region informationD1 includes, as coordinate information that defines the operation regionR1, information on coordinates C11 to C14 (see FIG. 2 ) of four cornersof the display screen 13A. The operation region information D1 isregistered in the storage 12 each time the operation region R1 is set orupdated.

The virtual operation plane information D2 is information on a region,which accepts the input operation by the user for the display screen13A, in the virtual operation plane R2. More specifically, the virtualoperation plane R2 corresponds to the operation region R1, andcoordinates C21 to C24 (see FIG. 2 ) of four corners that define thevirtual operation plane R2 respectively correspond to the coordinatesC11 to C14 that define the operation region R1. The virtual operationplane information D2 includes information on the coordinates C21 to C24of the four corners that define the virtual operation plane R2. Thevirtual operation plane information D2 is registered in the storage 12each time the virtual operation plane R2 is set or updated. Size and aposition of the virtual operation plane R2 may be set in advance or maybe set by the user's operation.

In addition, the storage 12 stores a control program such as a displaycontrol program for causing the controller 11 to execute display controlprocessing (see FIG. 7 ), which will be described below. For example,the display control program is recorded in a non-transitory manner in acomputer-readable recording medium such as a CD or a DVD, is read by areader (not illustrated) such as a CD drive or a DVD drive provided inthe display apparatus 1, and is stored in the storage 12. The displaycontrol program may be distributed from a cloud server and stored in thestorage 12.

The controller 11 includes control devices such as a CPU, ROM, and RAM.The CPU is a processor that executes various types of calculationprocessing. The ROM is a non-volatile storage that stores, in advance,control programs such as BIOS and OS to cause the CPU to execute thevarious types of the calculation processing. The RAM is a volatile ornon-volatile storage that stores the various types of the information,and is used as temporary storage memory (a workspace) for various typesof the processing executed by the CPU. The controller 11 causes the CPUto execute the various control programs, which are stored in the ROM orthe storage 12 in advance, to control the display apparatus 1.

Here, in the related art, a position on the display screen 13A (theinput position) that is instructed through the gesture operation by theuser's hand is identified on the basis of a position of his/her hand(the operation position). For example, the conventional displayapparatus identifies, as the input position, a position of the user'sfinger joint, a center (center of gravity) position of the entire regionwhere the user's hand during the gesture operation is projected, or thelike. However, this method causes such a problem that the input positionis changed in a direction unintended by the user in the case where theuser changes his/her operation from an operation to open his/her hand (afirst gesture operation) to an operation to close his/her hand (a secondgesture operation), for example. A description will hereinafter be madeon a specific example of this problem.

FIG. 3 to FIG. 5 are views for explaining the problem caused by therelated art. For example, selection button images such as “ORDER”,“RETURN”, and “OTHER” are shown as selection target images on a productorder screen illustrated in FIG. 3 . Then, when, on the virtualoperation plane R2, the user performs an operation (the first gestureoperation) to hold the palm of the right hand RH to the display screen13A and moves the right hand RH in vertical and horizontal directions(an X-direction and a Y-direction), an input operation icon M1 on thedisplay screen 13A moves according to the movement operation. Whenordering a desired product, in order to press the selection button image“ORDER” that is shown on the display screen 13A, the user performs anoperation (the second gesture operation) to close the right hand RH in astate where the input operation icon M1 overlaps the selection buttonimage “ORDER”. As a result, the user can perform an operation to select(order) the desired product.

Here, as illustrated in FIG. 4A, the conventional display apparatusidentifies, as the input position, a center (center of gravity) positionP0 (the operation position) of an entire region F1 where the user's hand(in an opened state) is projected. Thus, as illustrated in FIG. 4B, whenthe user closes his/her hand, the operation position moves from thecenter position P0 of the entire region F1 to a center position P1 of anentire region F2. Consequently, the input position on the display screen13A is also changed according to the movement of the operation position.A movement vector including a movement amount and a movement directionof the input position is calculated on the basis of a movement amount ofthe operation position (a movement amount from the center position P0 tothe center position P1).

Accordingly, for example, as illustrated in FIG. 5 , in the case wherethe user keeps the palm of the right hand RH open on the virtualoperation plane R2, moves the right hand RH in the vertical andhorizontal directions (the X-direction and the Y-direction), andperforms the operation to close the right hand RH in the state where theinput operation icon M1 overlaps the selection button image “ORDER”, theinput position possibly moves downward, and an operation to press theselection button image “RETURN” is possibly accepted. Just as described,in the related art, there is a problem that the input position moves inthe direction unintended by the user in conjunction with a change in thegesture operation by the user. On the contrary, as will be describedbelow, the display apparatus 1 according to the present embodiment canprevent the change in the input position in the direction unintended bythe user due to the change in the gesture operation by the user.

More specifically, as illustrated in FIG. 1 , the controller 11 includesvarious processing devices such as an operation plane setting device111, an input operation detector 112, an operation positiondetermination device 113, a display processor 114, a movement vectorcalculator 115, an input processor 116, a gesture operation changedetermination device 117, and a movement vector corrector 118. Here, thecontroller 11 functions as the operation plane setting device 111, theinput operation detector 112, the operation position determinationdevice 113, the display processor 114, the movement vector calculator115, the input processor 116, the gesture operation change determinationdevice 117, and the movement vector corrector 118 when the CPU executesthe various types of the processing according to the display controlprogram. Some or all of the processors provided in the controller 11 maybe constructed of an electronic circuit. The display control program maybe a program that causes the plural processors to function as thevarious processors.

The operation plane setting device 111 sets the virtual operation planeR2 that accepts the input operation by the user. For example, asillustrated in FIG. 2 , the operation plane setting device 111 sets thevirtual operation plane R2 at a position that is away from the displayscreen 13A by a predetermined distance in the Z-direction. The operationplane setting device 111 may set the virtual operation plane R2 in apreset size at a preset position, or may set the virtual operation planeR2, whose size corresponds to the user's operation, at a positionspecified by the user. Although details will be described below (see“Method for Setting Virtual Operation Plane”), for example, the user canset the virtual operation plane R2 in the desired size at the desiredposition by performing a predetermined gesture operation.

The input operation detector 112 detects the input operation thatincludes the predetermined gesture operation by the user. Morespecifically, the input operation detector 112 detects the inputoperation by the user on the virtual operation plane R2, which is set bythe operation plane setting device 111. For example, the input operationdetector 112 detects detection coordinates (the operation position) onthe virtual operation plane R2 on the basis of the detection informationacquired from the motion sensor 15, and calculates input coordinates(the input position) in the operation region R1 from the detectioncoordinates. The input operations include the gesture operation toexecute the predetermined input processing, a drawing operation to drawa hand-drawn image on the display screen 13A, and the like. The inputoperation detector 112 is an example of the gesture operation detectoraccording to the present disclosure.

Here, in the case where a ratio of the virtual operation plane R2 to theoperation region R1 is set as “W2:W1=H2:H1=a:b” (see FIG. 2 ), the inputoperation detector 112 can calculate input coordinates [dx, dy] byequations, dx=sx×b/a and dy=sy×b/a, on the basis of detectioncoordinates [sx, sy] on the virtual operation plane R2. Here, displayresolution [rx, ry] is Min[dx, dy]=[0, 0] and Max[dx, dy]=[dx, dy].

A description will herein be made on a specific example of a case wherethe input operation detector 112 detects the gesture operation.

For example, when desiring to move the input operation icon M1 (forexample, a mouse cursor image) shown on the display screen 13A, the userperforms the first gesture operation. The first gesture operation is theuser's operation to open the right hand RH (for example, an operation topick “paper” by the right hand RH for rock paper scissors) in front ofthe display screen 13A, for example. In addition, for example, whendesiring to select (click) the selection target with the input operationicon M1 shown on the display screen 13A, the user performs the secondgesture operation. The second gesture operation is the user's operationto close the right hand RH (for example, an operation to pick “rock” bythe right hand RH for rock paper scissors) in front of the displayscreen 13A, for example. The input operation detector 112 detects thefirst gesture operation and the second gesture operation.

The combination of the first gesture operation and the second gestureoperation is not limited to the above-described example. For example,the first gesture operation may be a posture of putting up an indexfinger of the right hand RH, and the second gesture operation may be aposture of closing the right hand RH. The first gesture operation may bea posture of raising the opened right hand RH vertically, and the secondgesture operation may be a posture of tilting the opened right hand RH90 degrees. The first gesture operation may be a posture of opening theright hand RH, and the second gesture operation may be a posture ofmaking a thumbs-up by the right hand RH or a posture of making an OKsign by the right hand RH.

The input operation detector 112 also detects a third gesture operationthat approximates (is similar to) the first gesture operation and thesecond gesture operation. The third gesture operation is, for example, agesture operation in the middle of a change from the first gestureoperation to the second gesture operation, and is a gesture of making ashape between the “paper” shape and the “stone” shape by the right handRH, for example.

The input operation detector 112 detects the first gesture operation,the second gesture operation, or the third gesture operation on thebasis of hand and finger information that includes at least one of theskeleton, the angle, and the position of the user's hand and fingers.The input operation detector 112 detects the first gesture operation,the second gesture operation, or the third gesture operation on thebasis of a degree of clenching Gn (will be described below) of the handin accordance with the hand and finger information. For example, theinput operation detector 112 sets a first threshold of the degree ofclenching Gn that corresponds to the first gesture operation in advance,and detects the first gesture operation when the detected degree ofclenching Gn is lower than the first threshold. In addition, forexample, the input operation detector 112 sets a second threshold (here,the first threshold<the second threshold) of the degree of clenching Gnthat corresponds to the second gesture operation in advance, and detectsthe second gesture operation when the detected degree of clenching Gn isequal to or higher than the second threshold. Furthermore, the inputoperation detector 112 detects the third gesture operation when thedetected degree of clenching Gn is equal to or higher than the firstthreshold and is lower than the second threshold.

The storage 12 stores, in advance, setting information in which thepredetermined gesture operation is associated with an operation contentcorresponding to the gesture operation. For example, the first gestureoperation and the third gesture operation are associated with themovement operation of the input operation icon M1, and the secondgesture operation is associated with the selection operation (the clickoperation) by the input operation icon M1. The predetermined gestureoperation may include a gesture operation (see “Method for SettingVirtual Operation Plane” below) for setting the virtual operation planeR2. In addition, the predetermined gesture operation may be setindividually for each user using the display apparatus 1.

The operation position determination device 113 determines the operationposition of the gesture operation, which is detected by the inputoperation detector 112, on the virtual operation plane R2 for thegesture operation. More specifically, the operation positiondetermination device 113 determines a reference point of the gestureoperation on the basis of a captured image of the gesture operation,which is captured by the motion sensor 15, and determines the operationposition on the basis of the position of the reference point. Theoperation position determination device 113 determines, as the referencepoint, a center of gravity position of the captured image of the gestureoperation, which is captured by the motion sensor 15. For example, theoperation position determination device 113 creates first hand andfinger information on at least one of the skeleton, the angle, and theposition of the user's hand and fingers on the basis of the capturedimage of the gesture operation, which is captured by the motion sensor15, and determines a position of the reference point on the basis of thefirst hand and finger information. The center position P0 illustrated inFIG. 4A and the center position P1 illustrated in FIG. 4B are examplesof the reference point.

The display processor 114 shows the various types of the information onthe display screen 13A of the display panel 13. For example, the displayprocessor 114 shows, on the display screen 13A, an image of the inputoperation icon M1 (the mouse cursor image), the images of the selectiontargets that can be selected by the user operating the input operationicon M1 (for example, the selection button images “ORDER”, “RETURN”, and“OTHER” in FIG. 3 ), and the like. In addition, the display processor114 shows the input operation icon M1 at the input position thatcorresponds to the operation position determined by the operationposition determination device 113.

The display processor 114 may show the input operation icon M1 in adifferent display mode according to a type of gesture operation. Forexample, in the case where the input operation detector 112 detects thefirst gesture operation by the user, the display processor 114 shows theinput operation icon M1 in a first display mode on the display screen13A. In addition, in the case where the input operation detector 112detects the second gesture operation by the user, the display processor114 shows the input operation icon M1 in a second display mode on thedisplay screen 13A. More specifically, as illustrated in FIG. 6 , in thecase where the input operation detector 112 detects the first gestureoperation (the posture of “paper”) by the user, the display processor114 shows the input operation icon M1 in first size (dimensions) and afirst color (white). Meanwhile, in the case where the input operationdetector 112 detects the second gesture operation (the posture of“stone”) by the user, the display processor 114 shows the inputoperation icon M1 in second size, which is smaller than the first size,and a second color (black).

In the case where the operation position of the gesture operation by theuser moves, the movement vector calculator 115 calculates the movementvector, which includes the movement direction and the movement amountin/with which the input position moves, on the basis of the movementamount of the operation position. For example, as illustrated in FIG. 4Aand FIG. 4B, in the case where the user changes the gesture operationfrom the state of opening his/her hand to the state of closing his/herhand, the movement vector calculator 115 calculates the movementdirection and the movement amount (the movement vector) of the inputposition on the basis of a movement direction and a movement amount ofthe reference point (the center positions P0, P1) that corresponds tothe operation position. Here, the movement vector calculator 115converts the movement vector at the operation position into the movementvector at the input position on the basis of the ratio of the virtualoperation plane R2 to the operation region R1 (see FIG. 2 ), forexample.

The input processor 116 executes the input processing at the inputposition that corresponds to the input operation by the user. The inputprocessor 116 executes first input processing at the input position atthe time when the input operation detector 112 detects the first gestureoperation by the user, and executes second input processing at the inputposition at the time when the input operation detector 112 detects thesecond gesture operation by the user. For example, in the case where theinput operation detector 112 detects the movement operation to move theinput operation icon M1 according to the first gesture operation, theinput processor 116 moves the input operation icon M1 on the displayscreen 13A. In addition, for example, in the case where the inputoperation detector 112 detects the selection operation to select theselection target by the input operation icon M1, the input processor 116executes processing according to the selection target on the displayscreen 13A. For example, in the case where the input operation detector112 detects the operation to select the selection button image “ORDER”on the display screen 13A, the input processor 116 executes orderprocessing that is assigned to the selection button image “ORDER”.Furthermore, in the case where the input operation detector 112 detectsthe drawing operation, the input processor 116 executes drawingprocessing to draw the hand-drawn image on the display screen 13A. Theinput processor 116 is an example of the input processor according tothe present disclosure.

The gesture operation change determination device 117 determines thechange from the first gesture operation to the second gesture operation.For example, the gesture operation change determination device 117determines a change from the state where the user's hand is opened tothe state where the user's hand is closed.

More specifically, the gesture operation change determination device 117calculates a degree of approximation between the gesture operation bythe user, which is detected by the input operation detector 112, (forexample, the third gesture operation) and each of the first gestureoperation (the state where the user's hand is opened) and the secondgesture operation (the state where the user's hand is closed), anddetermines the change from the first gesture operation to the secondgesture operation on the basis of the degree of approximation. Forexample, the gesture operation change determination device 117 createssecond hand and finger information on at least one of the skeleton, theangle, and the position of the user's hand and fingers on the basis ofthe captured image of the gesture operation by the user, which iscaptured by the motion sensor 15, (the third gesture operation), anddetermines the degree of approximation on the basis of the second handand finger information. For example, in the case where a time changeamount of the degree of approximation is larger than a predeterminedfirst reference value, the gesture operation change determination device117 determines that the gesture operation has been changed from thefirst gesture operation to the second gesture operation.

A description will herein be made on an example of a method fordetermining the degree of approximation with reference to FIG. 6 . Forexample, in the case where the input operation detector 112 detects thethird gesture operation by the user, the gesture operation changedetermination device 117 calculates the degree of approximation betweenthe third gesture operation and each of the first gesture operation andthe second gesture operation. The gesture operation change determinationdevice 117 calculates the degree of approximation on the basis of thesecond hand and finger information. The degree of approximation is anindex indicating the degree of clenching Gn of the user's hand. FIG. 6illustrates the degree of clenching Gn of the user's hand that ischanged along with a time change.

For example, as illustrated in FIG. 6 , the degree of clenching Gn isset to “0” (or 0 to 0.1) in the first gesture operation (the operationto open the hand), and the degree of clenching Gn is set to “1” (or 0.9to 1) in the second gesture operation (the operation to clench thehand). The storage 12 stores, in advance, information on the degrees ofclenching Gn corresponding to the first gesture operation and the secondgesture operation, respectively. Here, when the input operation detector112 detects the third gesture operation by the user, the gestureoperation change determination device 117 calculates the degree ofclenching Gn, which corresponds to the third gesture operation, on thebasis of the second hand and finger information on at least one of theskeleton, the angle, and the position of the user's hand and fingers. Itis indicated that the third gesture operation approximates the firstgesture operation as the degree of clenching Gn becomes closer to “0”,and that the third gesture operation approximates the second gestureoperation as the degree of clenching Gn becomes closer to “1”. Based onthe degree of clenching Gn that corresponds to the third gestureoperation, the gesture operation change determination device 117calculates a degree of approximation A1 between the first gestureoperation and the third gesture operation and a degree of approximationA2 between the second gesture operation and the third gesture operation.The gesture operation change determination device 117 determines thechange from the first gesture operation to the second gesture operationon the basis of the degrees of approximation A1, A2. In addition, in thecase where time change amounts of the degrees of approximation A1, A2are larger than the predetermined first reference value, the gestureoperation change determination device 117 determines that the gestureoperation has been changed from the first gesture operation to thesecond gesture operation.

The movement vector corrector 118 corrects the movement vector in thecase where the gesture operation change determination device 117determines the change from the first gesture operation to the secondgesture operation.

For example, the movement vector corrector 118 corrects the movementvector by subtracting a predetermined correction movement amount, whichis set in advance, from the movement amount calculated by the movementvector calculator 115. Alternatively, for example, the movement vectorcorrector 118 corrects the movement vector by subtracting a correctionmovement vector, which includes a correction movement direction and acorrection movement amount set in advance, from the movement vectorcalculated by the movement vector calculator 115. For example, pluralcombinations of the movement amount of the operating position of thegesture operation by the user and the correction movement amountcorresponding to the movement amount may be registered in the storage 12in advance. The movement vector corrector 118 refers to the storage 12to acquire the correction movement amount that corresponds to themovement amount calculated by the movement vector calculator 115, andcorrects the movement vector.

As another embodiment, the movement vector corrector 118 may correct themovement vector by multiplying the movement amount, which is calculatedby the movement vector calculator 115, by a predetermined correctioncoefficient as a smaller value than 1, which is set in advance. Forexample, plural combinations of the movement amount of the operatingposition of the gesture operation by the user and t the correctioncoefficient corresponding to the movement amount may be registered inthe storage 12 in advance. The movement vector corrector 118 refers tothe storage 12 to acquire the correction coefficient that corresponds tothe movement amount calculated by the movement vector calculator 115,and corrects the movement vector.

In addition, as another embodiment, the movement vector corrector 118may correct the movement vector such that the movement amount becomesequal to or smaller than a predetermined upper limit value, which is setin advance, when the movement amount calculated by the movement vectorcalculator 115 is larger than the upper limit value. The upper limit maybe registered in the storage 12 in advance. The movement vectorcorrector 118 executes correction processing to correct the movementvector in the case where the movement amount calculated by the movementvector calculator 115 is larger than the upper limit value. According tosuch a configuration, the correction processing is executed only whennecessary. Thus, a processing load can be reduced.

As described above, the movement vector corrector 118 corrects themovement vector in a manner to offset the movement of the inputposition, which occurs when the reference point is changed from aposition of the reference point (the center position P0 in FIG. 4A) forthe first gesture operation to a position of the reference point (thecenter position P1 in FIG. 4B) for the second gesture operation.

Here, in the case where the gesture operation change determinationdevice 117 is configured to determine that the gesture operation hasbeen changed from the first gesture operation to the second gestureoperation when the time change amount of the degree of approximation islarger than the predetermined first reference value, the movement vectorcorrector 118 may correct the movement vector until predeterminedduration elapses from a time point at which it is determined that thetime change amount of the degree of approximation is larger than thefirst reference value. In this way, the execution of the correctionprocessing can be limited to the duration. Thus, the processing load canbe reduced. The duration may be set in advance in a manner to correspondto a magnitude of the degree of approximation.

The movement vector corrector 118 may correct the movement vector from atime point at which it is determined that the time change amount of thedegree of approximation is larger than the first reference value to atime point at which it is determined that the time change amount of thedegree of approximation is equal to or smaller than the first referencevalue. In this way, it is possible to reliably reduce the movementamount of the movement vector to a predetermined amount.

In the case where the time change amount of the degree of approximationis larger than a predetermined second reference value, the movementvector corrector 118 may enhance a correction effect to correct themovement vector in comparison with a case where the time change amountof the degree of approximation is smaller than the second referencevalue. In this way, the movement vector can be corrected according to adegree of change in the gesture operation. Thus, the correctionprocessing can be executed efficiently.

As described above, the position of the reference point can be fixed byexecuting the correction processing by the movement vector corrector 118regardless of the change in the gesture operation by the user. Thus, itis possible to prevent the movement of the input position caused by thechange in the gesture operation.

Display Control Processing

A description will hereinafter be made on the display control processingthat is executed by the controller 11 of the display apparatus 1 withreference to FIG. 7 .

The present disclosure can be regarded as disclosure of a displaycontrol method (an example of the input method in the presentdisclosure) for executing one or plural steps included in the displaycontrol processing, and one or plural steps included in the displaycontrol processing described herein may appropriately be omitted. Thesteps in the display control processing may be executed in a differentorder as long as a similar operational effect is exerted. Furthermore, adescription will herein be made on a case where the controller 11executes the steps in the display control processing as an example.However, the display control method in which the plural processorsseparately execute the steps in the display control processing isconsidered as another embodiment.

First, in step S11, the controller 11 determines whether the inputoperation by the user has been detected. More specifically, thecontroller 11 detects the input operation by the user on the virtualoperation plane R2. For example, the controller 11 detects the detectioncoordinates on the virtual operation plane R2 on the basis of thedetection information acquired from the motion sensor 15, and calculatesthe input coordinates in the operation region R1 of the display screen13A from the detection coordinates. If the controller 11 has detectedthe input operation (S11: Yes), the processing proceeds to step S12. Ifthe controller 11 has not detected the input operation (S11: No), theprocessing proceeds to step S18.

In step S12, the controller 11 determines whether the predeterminedgesture operation by the user has been detected. For example, thecontroller 11 detects the operation to open the right hand RH (theoperation to pick “paper” by the right hand RH) by the user or theoperation to close the right hand RH (the operation to pick “rock” bythe right hand RH) by the user on the basis of the hand and fingerinformation on at least one of the skeleton, the angle, and the positionof the user's hand and fingers. If the controller 11 has detected thegesture operation by the user (S12: Yes), the processing proceeds tostep S13. On the other hand, if the controller 11 has not detected thegesture operation by the user (S12: No), the processing proceeds to stepS17. Step S12 is an example of the gesture operation detection in thepresent disclosure.

Next, in step S13, the controller 11 determines the operation positionof the gesture operation. More specifically, the controller 11determines the reference point for the gesture operation on the virtualoperation plane R2 on the basis of the captured image of the gestureoperation, which is captured by the motion sensor 15, and thendetermines the operation position on the basis of the position of thereference point. For example, the controller 11 creates the first handand finger information on at least one of the skeleton, the angle, andthe position of the user's hand and fingers on the basis of the capturedimage of the gesture operation, which is captured by the motion sensor15, determines the position of the reference point (for example, thecenter position P0 illustrated in FIG. 4A) on the basis of the firsthand and finger information, and determines the operation position onthe basis of the position of the reference point. Step S13 is an exampleof the operation position determination in the present disclosure.

In addition, in step S13, the controller 11 shows the input operationicon M1 at the input position on the display screen 13A that correspondsto the operation position.

Next, in step S14, the controller 11 executes the input processing thatcorresponds to the gesture operation by the user. For example, when theuser moves the right hand RH in the vertical and horizontal directions(the X-direction and the Y-direction) in the state of opening the righthand RH (the first gesture operation) on the virtual operation plane R2,the controller 11 accepts the movement operation and moves the inputoperation icon M1, which is shown on the display screen 13A, in a mannerto follow the movement of the user's right hand RH. Step S14 is anexample of the inputting in the present disclosure.

In step S14, in the case where the operation position of the gestureoperation by the user moves, the controller 11 calculates the movementvector, which includes the movement direction and the movement amountin/with which the input position moves, on the basis of the movementamount of the operation position. The controller 11 calculates themovement vector at the input position on the basis of the movementvector at the operation position and moves the input operation icon M1on the display screen 13A. Step S14 is an example of the movement vectorcalculation in the present disclosure.

Next, in step S15, the controller 11 determines whether the gestureoperation has been changed. More specifically, the controller 11calculates the degree of approximation between the detected gestureoperation by the user and each of the first gesture operation (the statewhere the user's hand is opened) and the second gesture operation (thestate where the user's hand is closed), and determines the change fromthe first gesture operation to the second gesture operation on the basisof the degree of approximation. For example, in the case where the timechange amount of the degree of approximation is larger than thepredetermined first reference value, the controller 11 determines thatthe detected gesture operation has been changed from the first gestureoperation to the second gesture operation. If the gesture operation hasbeen changed (S15: Yes), the processing proceeds to step S16. If thegesture operation has not been changed (S15: No), the processingproceeds to step S18. Step S15 is an example of the gesture operationchange determination in the present disclosure.

Next, in step S16, the controller 11 corrects the movement vector of theinput position. For example, the controller 11 corrects the movementvector by subtracting the predetermined correction movement amount,which is set in advance, from the movement amount of the input positionthat corresponds to the movement amount of the operation position of thegesture operation by the user.

Alternatively, the controller 11 may correct the movement vector bymultiplying the calculated movement amount by the predeterminedcorrection coefficient as the smaller value than 1, which is set inadvance. In addition, the controller 11 may correct the movement vectorsuch that the movement amount becomes equal to or smaller than thepredetermined upper limit value, which is set in advance, when thecalculated movement amount is larger than the upper limit value.

As described above, the controller 11 corrects the movement vector inthe manner to offset the movement of the input position, which occurswhen the reference point is changed from the position of the referencepoint (the center position P0 in FIG. 4A) for the first gestureoperation to the position of the reference point (the center position P1in FIG. 4B) for the second gesture operation. Step S16 is an example ofthe movement vector correction in the present disclosure.

Next, in step S17, the controller 11 executes the input processing thatcorresponds to the gesture operation by the user. For example, thecontroller 11 executes the input processing with the input operationicon M1 on the basis of the corrected movement vector. For example, whenthe controller 11 determines the change from the first gesture operation(the state where the user's hand is opened) to the second gestureoperation (the state where the user's hand is closed) at the positionwhere the input operation icon M1 overlaps the selection button image“ORDER” on the display screen 13A, the controller 11 offsets the changein the input position and executes the order processing that is assignedto the selection button image “ORDER”.

Next, in step S18, the controller 11 determines whether the inputoperation has been finished. If the input operation has been finished(S18: Yes), the controller 11 finishes the display control processing.If the input operation has not been finished (S18: No), the processingreturns to step S11.

The controller 11 executes the display control processing as describedso far.

As it has been described so far, the display apparatus 1 according tothe present embodiment detects the gesture operation by the user anddetermines the operation position of the detected gesture operation. Inaddition, in the case where the operation position has moved, thedisplay apparatus 1 calculates the movement vector, which includes themovement direction and the movement amount in/with which the inputposition moves, on the basis of the movement amount of the operationposition. The display apparatus 1 executes the first input processing atthe input position at the time when the first gesture operation by theuser is detected, and executes the second input processing at the inputposition at the time when the second gesture operation by the user isdetected. The display apparatus 1 corrects the movement vector in thecase where the change from the first gesture operation to the secondgesture operation is determined. More specifically, the displayapparatus 1 corrects the movement vector in the manner to offset themovement of the input position that is associated with the movement ofthe reference point for the gesture operation.

In this way, it is possible to prevent the change in the input positionin the direction unintended by the user in the case where the userchanges his/her operation from the operation to open his/her hand (thefirst gesture operation) to the operation to close (clench) his/her hand(the second gesture operation), for example.

Method for Setting Virtual Operation Plane

The operation plane setting device 111 may set the virtual operationplane R2 in the desired size as the desired position on the basis of apredetermined setting gesture operation by the user.

More specifically, the operation plane setting device 111 detects thesetting gesture operation by the user. More specifically, the operationplane setting device 111 detects the setting gesture operation on thebasis of the detection information acquired from the motion sensor 15.For example, the operation plane setting device 111 identifies theuser's hand shape on the basis of the coordinate information included inthe detection information, and identifies the corresponding gestureoperation of the plural gesture operations, which are registered in thestorage 12 in advance.

When detecting a predetermined first setting gesture operation by theuser the operation plane setting device 111 sets the regioncorresponding to the first setting gesture operation as the virtualoperation plane R2 for accepting the input operation by the user for thedisplay screen 13A. The operation plane setting device 111 may set thevirtual operation plane R2 in the case where the first setting gestureoperation continues for a predetermined period of time. The firstsetting gesture operation is an operation to hold the palm of each ofthe left hand LH and the right hand RH toward the display screen 13A,for example. That is, the first setting gesture operation is a settingoperation to set the virtual operation plane R2 by the user.

For example, as illustrated in FIG. 8 , when the user holds the palm ofthe left hand LH at an upper right position toward the display screen13A and holds the palm of the right hand RH at a lower right positiontoward the display screen 13A, the operation plane setting device 111detects coordinates Ph1 of the left hand LH, the coordinates Ph2 of theright hand RH, and the first setting gesture operation to hold the lefthand LH and the right hand RH on the basis of the detection informationacquired from the motion sensor 15. When detecting the first settinggesture operation, the operation plane setting device 111 sets thevirtual operation plane R2 on the basis of the coordinates Ph1 of theleft hand LH and the coordinates Ph2 of the right hand RH.

For example, as illustrated in FIG. 8 , the operation plane settingdevice 111 sets the rectangular virtual operation plane R2 having, as adiagonal line, a line connecting the position of the left hand LH (thecoordinates Ph1) and the position of the right hand RH (the coordinatesPh2). More specifically, the operation plane setting device 111calculates the coordinates C21 to C24 (see FIG. 2 ) of corners of therectangle on the basis of the coordinates Ph1 of the left hand LH andthe coordinates Ph2 of the right hand RH, and sets the virtual operationplane R2.

For example, the operation plane setting device 111 sets the virtualoperation plane R2 at a position that is away from the display screen13A by a predetermined distance L1. The predetermined distance L1 is adistance that corresponds to the coordinate Ph1 (the Z-coordinate) ofthe left hand LH and the coordinate Ph2 (the Z-coordinate) of the righthand RH.

For example, the operation plane setting device 111 may set the virtualoperation plane R2 whose aspect ratio is the same as an aspect ratio ofthe display screen 13A. More specifically, as illustrated in FIG. 9 ,the operation plane setting device 111 sets such a virtual operationplane R2 that the aspect ratio (H1:W1) of the display screen 13A and theaspect ratio (H2:W2) of the virtual operation plane R2 are equal to eachother (H1:W1=H2:W2).

As described above, the size of the display screen 13A (the operationregion R1) and the size of the virtual operation plane R2 may be thesame or differ from each other. Here, the virtual operation plane R2that is smaller than the operation region R1 is suited for suchapplication that the large-sized display panel 13 is operated at theuser's hand. On the contrary, the virtual operation plane R2 that islarger than the operation region R1 is suited for such application thatthe small-sized display panel 13 is operated at a distance position.

As illustrated in FIG. 10 , the operation plane setting device 111 mayset the virtual operation plane R2 having a predetermined angle d1 thatis not parallel to the display screen 13A. That is, the virtualoperation plane R2 may be set in an oblique direction with respect tothe display screen 13A. For example, the operation plane setting device111 sets the predetermined angle d1 on the basis of the coordinate Ph1(the Z-coordinate) of the left hand LH and the coordinate Ph2 (theZ-coordinate) of the right hand RH. In this way, the user can performthe input operation in the oblique direction with respect to the displayscreen 13A. The operation plane setting device 111 may show informationon the predetermined angle d1 on the display screen 13A. In this way,the user can comprehend the angle (a degree of the inclination) of thevirtual operation plane R2 with respect to the display screen 13A.

The operation plane setting device 111 may set the virtual operationplane R2 that corresponds to the partial region of the display screen13A. For example, as illustrated in FIG. 11 , the operation planesetting device 111 sets the virtual operation plane R2 that correspondsto the operation region R1 as a part (a left region) of the displayscreen 13A. A position and size of the operation region R1 can be set bythe setting operation by the user. Here, in order for the user who setsthe virtual operation plane R2 to easily comprehend the operation regionR1, as illustrated in FIG. 12 , the operation plane setting device 111may show an object image T1 indicating the operation region R1 on thedisplay screen 13A at the time of setting the virtual operation planeR2.

The operation plane setting device 111 can use well-known coordinatetransformation (projective transformation, affine transformation, or thelike) to set the virtual operation plane R2, which is associated withthe operation region R1 of the display screen 13A, on the basis of thecoordinates corresponding to the first setting gesture operation.

The operation plane setting device 111 may execute processing to adjustthe set virtual operation plane R2. More specifically, in the case wherea predetermined second setting gesture operation by the user is detectedafter the virtual operation plane R2 is set, the operation plane settingdevice 111 changes at least one of the size and the position of thevirtual operation plane R2 on the basis of the second setting gestureoperation. The second setting gesture operation is a finger pointingoperation (see FIG. 13 ) by the right hand RH, for example.

For example, as illustrated in FIG. 13 , in the case where the userperforms the finger pointing operation by the right hand RH to point thedisplay screen 13A after the virtual operation plane R2 is set, theoperation plane setting device 111 detects coordinates Ph3 of the righthand RH and the second setting gesture operation that is the fingerpointing operation by the right hand RH on the basis of the detectioninformation acquired from the motion sensor 15. When detecting thesecond setting gesture operation, the operation plane setting device 111sets the virtual operation plane R2 to be movable on the basis of thecoordinates Ph3 of the right hand RH, and accepts the movement operationof the virtual operation plane R2 by the user. For example, when theuser moves the right hand RH to the left while keeping the fingerpointing state, the operation plane setting device 111 moves the virtualoperation plane R2 to the left by an amount corresponding to themovement amount of the right hand RH. That is, the operation planesetting device 111 sets the virtual operation plane R2 at thecoordinates Ph3 of the right hand RH that has moved.

For example, as illustrated in FIG. 14 , in the case where the userperforms the operation to clench the right hand RH while holding theleft hand LH after the virtual operation plane R2 is set, the operationplane setting device 111 detects the coordinates Ph1 of the left handLH, the coordinates Ph2 of the right hand RH, and the second settinggesture operation to hold the left hand LH and clench the right hand RHon the basis of the detection information acquired from the motionsensor 15. When detecting the second setting gesture operation, theoperation plane setting device 111 sets the size of the virtualoperation plane R2 to be changeable on the basis of the coordinates Ph2of the right hand RH, and accepts the user's operation to change thesize of the virtual operation plane R2. For example, when the user movesthe right hand RH in a lower-right direction while clenching the righthand RH, the operation plane setting device 111 enlarges the size (anarea) of the virtual operation plane R2 by an amount corresponding tothe movement amount of the right hand RH. That is, the operation planesetting device 111 sets the virtual operation plane R2 that is definedby the coordinates Ph1 of the left hand LH and the coordinates Ph2 ofthe right hand RH that has moved.

FIG. 15 illustrates an example of the case where the user performs theoperation to clench the left hand LH while holding the right hand RHafter the virtual operation plane R2 is set. In this case, the operationplane setting device 111 detects the coordinates Ph1 of the left handLH, the coordinates Ph2 of the right hand RH, and the second settinggesture operation to hold the right hand RH and clench the left hand LHon the basis of the detection information acquired from the motionsensor 15. When detecting the second setting gesture operation, theoperation plane setting device 111 sets the size of the virtualoperation plane R2 to be changeable on the basis of the coordinates Ph1of the left hand LH, and accepts the user's operation to change the sizeof the virtual operation plane R2. For example, when the user moves theleft hand LH in the lower-right direction while clenching the left handLH, the operation plane setting device 111 reduces the size (the area)of the virtual operation plane R2 by the amount corresponding to themovement amount of the left hand LH. That is, the operation planesetting device 111 sets the virtual operation plane R2 that is definedby the coordinates Ph2 of the right hand RH and the coordinates Ph1 ofthe left hand LH that has moved.

In the case where the second setting gesture operation is detected afterthe virtual operation plane R2 is set, the operation plane settingdevice 111 may show an object image T2 indicating the virtual operationplane R2 on the display screen 13A in accordance with the second settinggesture operation. FIG. 16 illustrates an example of the object image T2that indicates the virtual operation plane R2 after the size thereof ischanged. According to such a configuration, the user can visuallycomprehend the size, the position, and the like of the virtual operationplane R2 that has changed.

It is to be understood that the embodiments herein are illustrative andnot restrictive, since the scope of the disclosure is defined by theappended claims rather than by the description preceding them, and allchanges that fall within metes and bounds of the claims, or equivalenceof such metes and bounds thereof are therefore intended to be embracedby the claims.

What is claimed is:
 1. An input apparatus that detects a gestureoperation by a user and executes input processing with respect to aninput position on a display screen, the input apparatus comprising: agesture operation detector that detects the gesture operation by theuser; an operation position determination device that determines anoperation position of the gesture operation detected by the gestureoperation detector; a movement vector calculator that calculates amovement vector to move the input position on the basis of a movementamount of the operation position when the operation position moves, themovement vector including a movement direction and a movement amount; aninput processor that executes first input processing at the inputposition at the time when the gesture operation detector detects a firstgesture operation by the user, and executes second input processing atthe input position at the time when the gesture operation detectordetects a second gesture operation by the user; a gesture operationchange determination device that determines a change from the firstgesture operation to the second gesture operation; and a movement vectorcorrector that corrects the movement vector in the case where thegesture operation change determination device determines the change fromthe first gesture operation to the second gesture operation.
 2. Theinput apparatus according to claim 1, wherein the movement vectorcorrector corrects the movement vector by subtracting a predeterminedcorrection movement amount, which is set in advance, from the movementamount calculated by the movement vector calculator.
 3. The inputapparatus according to claim 1, wherein the movement vector correctorcorrects the movement vector by subtracting a correction movementvector, which is set in advance, from the movement vector calculated bythe movement vector calculator, the correction movement vector includinga correction movement direction and a correction movement amount.
 4. Theinput apparatus according to claim 1, wherein the movement vectorcorrector corrects the movement vector by multiplying the movementamount, which is calculated by the movement vector calculator, by apredetermined correction coefficient, which is set in advance, thepredetermined correction coefficient being a smaller value than
 1. 5.The input apparatus according to claim 1, wherein in the case where themovement amount calculated by the movement vector calculator is largerthan a predetermined upper limit value, which is set in advance, themovement vector corrector corrects the movement vector such that themovement amount becomes equal to or smaller than the upper limit value.6. The input apparatus according to claim 1 further comprising: animager that captures an image of the gesture operation, wherein theoperation position determination device determines a reference point forthe gesture operation on the basis of a captured image of the gestureoperation, which is captured by the imager, and determines the operationposition on the basis of a position of the reference point.
 7. The inputapparatus according to claim 6, wherein the operation positiondetermination device determines, as the reference point, a center ofgravity position of the captured image of the gesture operation, whichis captured by the imager.
 8. The input apparatus according to claim 6,wherein the operation position determination device creates first handand finger information on at least one of a skeleton, an angle, and aposition of the user's hand and fingers on the basis of the capturedimage of the gesture operation, which is captured by the imager, anddetermines a position of the reference point on the basis of the firsthand and finger information.
 9. The input apparatus according to claim6, wherein the movement vector corrector corrects the movement vector ina manner to offset movement of the input position, which occurs when thereference point is changed from a position of the reference point forthe first gesture operation to a position of the reference point for thesecond gesture operation.
 10. The input apparatus according to claim 1,wherein the gesture operation change determination device calculates adegree of approximation between the gesture operation by the user, whichis detected by the gesture operation detector, and each of the firstgesture operation and the second gesture operation, and determines thechange from the first gesture operation to the second gesture operationon the basis of the degree of approximation.
 11. The input apparatusaccording to claim 10 further comprising: an imager that captures animage of the gesture operation, wherein the gesture operation changedetermination device creates second hand and finger information on atleast one of a skeleton, an angle, and a position of the user's hand andfingers on the basis of the captured image of the gesture operation bythe user, which is captured by the imager, and determines the degree ofapproximation on the basis of the second hand and finger information.12. The input apparatus according to claim 10, wherein in the case wherea time change amount of the degree of approximation is larger than apredetermined first reference value, the gesture operation changedetermination device determines that the gesture operation has beenchanged from the first gesture operation to the second gestureoperation.
 13. The input apparatus according to claim 12, wherein themovement vector corrector corrects the movement vector untilpredetermined duration elapses from a time point at which it isdetermined that the time change amount of the degree of approximation islarger than the first reference value.
 14. The input apparatus accordingto claim 13, wherein the movement vector corrector corrects the movementvector from the time point at which it is determined that the timechange amount of the degree of approximation is larger than the firstreference value to a time point at which it is determined that the timechange amount of the degree of approximation is equal to or smaller thanthe first reference value.
 15. The input apparatus according to claim12, wherein in the case where the time change amount of the degree ofapproximation is larger than a predetermined second reference value, themovement vector corrector enhances a correction effect to correct themovement vector in comparison with a case where the time change amountof the degree of approximation is smaller than the second referencevalue.
 16. An input method for detecting a gesture operation by a userand executing input processing with respect to an input position on adisplay screen, the method causing one or plural processors to: detectthe gesture operation by the user; determine an operation position ofthe detected gesture operation; calculate a movement vector to move theinput position on the basis of a movement amount of the operationposition when the operation position moves, the movement vectorincluding a movement direction and a movement amount; execute firstinput processing at the input position at the time of detecting a firstgesture operation by the user and execute second input processing at theinput position at the time of detecting a second gesture operation bythe user; determine a change from the first gesture operation to thesecond gesture operation; and correct the movement vector whendetermining the change from the first gesture operation to the secondgesture operation.
 17. A non-transitory computer-readable recordingmedium that records an input program for detecting a gesture operationby a user and executing input processing with respect to an inputposition on a display screen, the non-transitory computer-readablerecording medium recording the input program to cause one or pluralprocessors to: detect the gesture operation by the user; determine anoperation position of the detected gesture operation; calculates amovement vector to move the input operation on the basis of a movementamount of the operation position when the operation position moves, themovement vector including a movement direction and a movement amount;execute first input processing at the input position at the time ofdetecting a first gesture operation by the user and execute second inputprocessing at the input position at the time of detecting a secondgesture operation by the user; determine a change from the first gestureoperation to the second gesture operation; and correct the movementvector when determining the change from the first gesture operation tothe second gesture operation.